Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Updated to collect dpkg.log and verify installed files. Also, search for package name that contains installed files. #253

Merged
merged 6 commits into from
Dec 3, 2024
Merged

Updated to collect dpkg.log and verify installed files. Also, search for package name that contains installed files. #253

merged 6 commits into from
Dec 3, 2024

Conversation

mnrkbys
Copy link
Contributor

@mnrkbys mnrkbys commented Jul 22, 2024 •
edited
Loading

Added an artifact to collect /var/log/dpkg.log to artifacts/files/packages/dpkg.yaml.
Added an artifact to verify files installed from dpkg package. This is similar to the "rpm -q -a" command in artifacts/live_response/packages/dpkg.yaml.

And I also added artifacts that search for packages that contain installed files. Files not included in a package may have been installed manually, helping to find suspect files.
As you know, /bin is a symbolic link to /usr/bin. Therefore, whether the installation destination directory is /bin or /usr/bin, the files are installed in the same directory.
However, "dpkg -S /bin/ls" and "dpkg -S /usr/bin/ls" return different results. Therefore, "dpkg -S" must be executed for both file paths.

"dpkg -S /bin/ls" vs "dpkg -S /usr/bin/ls"

$ dpkg -S /bin/ls
coreutils: /bin/ls
$ dpkg -S /usr/bin/ls
dpkg-query: no path found matching pattern /usr/bin/ls

But, "rpm -q -f" will returns same results.
"rpm -q -f /bin/ls" vs "rpm -q -f /usr/bin/ls"

$ rpm -q -f /bin/ls
coreutils-8.32-35.el9.x86_64
$ rpm -q -f /usr/bin/ls
coreutils-8.32-35.el9.x86_64

@mnrkbys
artif: new artifacts related to dpkg and rpm
d1d2e1d 
Updated to collect dpkg.log and verify installed files. Also, search for package name that contains installed files.
@mnrkbys mnrkbys force-pushed the rpm-dpkg-search-filename branch from aecea77 to d1d2e1d Compare July 23, 2024 02:01
@tclahr
Copy link
Owner

tclahr commented Aug 24, 2024

Do you know if we can do the same for additional package managers such as pacman, brew, zipper...?

@mnrkbys
Copy link
Contributor Author

mnrkbys commented Aug 26, 2024

I've tried zypper on openSUSE, and the results are like below.

> zypper --quiet search --provides --match-exact --installed-only /usr/bin/ls

S  | Name      | Summary            | Type
---+-----------+--------------------+--------
i  | coreutils | GNU Core Utilities | package
> zypper --quiet search --provides --match-exact --installed-only /bin/ls

S  | Name      | Summary            | Type
---+-----------+--------------------+--------
i  | coreutils | GNU Core Utilities | package
> zypper --quiet search --provides --match-exact --installed-only /nonexistent/file
No matching items found.

I believe zypper can check file paths as well as dpkg/rpm.
However, files under /usr/sbin/ will show different results for the paths /sbin/ and /usr/sbin/.

> zypper --quiet search --provides --match-exact --installed-only /sbin/bpftool 
No matching items found.
> zypper --quiet search --provides --match-exact --installed-only /usr/sbin/bpftool 

S  | Name    | Summary                                                       | Type
---+---------+---------------------------------------------------------------+--------
i+ | bpftool | Tool for inspection and manipulation of BPF programs and maps | package

@mnrkbys
Copy link
Contributor Author

mnrkbys commented Aug 26, 2024

I have found a useful table.
This includes package management commands for various distributions (except brew).

pacman/Rosetta - ArchWiki
https://wiki.archlinux.org/title/Pacman/Rosetta

@mnrkbys
Copy link
Contributor Author

mnrkbys commented Aug 26, 2024

As for brew, it does not seem to have the same functionality.
However, you can get a list of files included in a package with list -v <package_name>, so we may be able to find files not included in installed packages by comparing the list with files under /opt/homebrew/bin.

mnrkbys and others added 2 commits September 18, 2024 16:53
@tclahr
Copy link
Owner

tclahr commented Nov 23, 2024

I did some tests on some debian (dpkg), redhat (rpm) and arch (pacman) based systems and the collection time was increased by approximately 10 to 15 min in some situations.
The command to check whether a file belongs to an installed package is very costly (consumes a lot of resources).

Also, it seems that dpkg -S reports a lot of false-positives. Look at the samples below.

From Ubuntu 22.04:
$ cd /bin && for file in *; do dpkg -S "${file}"; done 2>&1 | grep "no path found"

dpkg-query: no path found matching pattern [
dpkg-query: no path found matching pattern *gnome-text-editor*
dpkg-query: no path found matching pattern *gnome-www-browser*
dpkg-query: no path found matching pattern *gstreamer-codec-install*
dpkg-query: no path found matching pattern *i686-linux-gnu-pkg-config*
dpkg-query: no path found matching pattern *infobrowser*
dpkg-query: no path found matching pattern *lzcat*
dpkg-query: no path found matching pattern *lzcmp*
dpkg-query: no path found matching pattern *lzdiff*
dpkg-query: no path found matching pattern *lzegrep*
dpkg-query: no path found matching pattern *lzfgrep*
dpkg-query: no path found matching pattern *lzgrep*
dpkg-query: no path found matching pattern *lzless*
dpkg-query: no path found matching pattern *lzmore*
dpkg-query: no path found matching pattern *nawk*
dpkg-query: no path found matching pattern *pinentry-x11*
dpkg-query: no path found matching pattern *test123*
dpkg-query: no path found matching pattern *x86_64-linux-gnu-pkg-config*
dpkg-query: no path found matching pattern *x-session-manager*
dpkg-query: no path found matching pattern *x-terminal-emulator*
dpkg-query: no path found matching pattern *x-www-browser*

During my tests I did not notice lots of false positives for rpm -q -f on Fedora/Red Hat based systems, but as most binaries were not installed by rpm in AIX, it does report lots of false-positives.

From AIX 7.3:
$ cd /usr/bin && for file in *; do rpm -q -f "${file}" | sed "s|$|: ${file}|"; done | grep "not owned"

file /usr/bin/2to3 is not owned by any package: 2to3
file /usr/bin/2to3-3.9 is not owned by any package: 2to3-3.9
file /usr/bin/IbDebugChk is not owned by any package: IbDebugChk
file /usr/bin/IbDebugOff is not owned by any package: IbDebugOff
file /usr/bin/IbDebugOn is not owned by any package: IbDebugOn
file /usr/bin/Mail is not owned by any package: Mail
file /usr/bin/ManGetURL.class is not owned by any package: ManGetURL.class
file /usr/bin/Rsh is not owned by any package: Rsh
file /usr/bin/SpmiArmd is not owned by any package: SpmiArmd
file /usr/bin/SpmiResp is not owned by any package: SpmiResp
file /usr/bin/X11 is not owned by any package: X11
file /usr/bin/X11r5 is not owned by any package: X11r5
file /usr/bin/X11r6 is not owned by any package: X11r6
file /usr/bin/[ is not owned by any package: [
file /usr/bin/acctcom is not owned by any package: acctcom
file /usr/bin/acctctl is not owned by any package: acctctl
file /usr/bin/acctras is not owned by any package: acctras
file /usr/bin/acctrpt is not owned by any package: acctrpt
file /usr/bin/aclconvert is not owned by any package: aclconvert
...
...

As the hash_executables.yaml artifact can be used to identify malicious files, I will create a different yaml file for this, but not add it to any profile. It will be available to be run as needed. I will commit to this PR and send it for your review.

Thanks a lot!

@mnrkbys
Copy link
Contributor Author

mnrkbys commented Nov 26, 2024

Thanks for testing this.
Indeed, I should have mentioned that the command takes a long time to execute.

@tclahr
artif: new artifact
8bceef6 
Added collection of which installed package owns a specific file or command. Note that this artifact is resource-intensive and time-consuming to execute, so it is disabled by default in all profiles
@tclahr
Copy link
Owner

tclahr commented Nov 27, 2024

@mnrkbys can you review the changes, please?

@mnrkbys
Copy link
Contributor Author

mnrkbys commented Dec 2, 2024

Thanks @tclahr
It appears to me that it is not an issue.

@tclahr tclahr merged commit fc4593a into tclahr:develop Dec 3, 2024
2 checks passed
@mnrkbys mnrkbys deleted the rpm-dpkg-search-filename branch December 9, 2024 02:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@mnrkbys @tclahr